High-resolution microscopy suffers from limited depth of field, which prevents thick or uneven specimen preparations from being imaged entirely in focus. Objects that appear outside the narrow depth of field or focal plane become quickly blurred and out of focus, forcing the microscopist to constantly manually focus back and forth. This not only limits the productivity of the microscopist but also increases the likelihood he or she will miss a subtle feature that may appear only in a narrow focal plane.
Moreover, this limited depth of field becomes worse as the magnification increases because it is directly dependent on the numerical aperture of the microscope objective, which increases with the magnification. This prevents a microscopist from using the highest magnification objective on a consistent basis, and forces him/her to strike a balance between magnification and the labor-intensive task of constant up and down focusing.
Thick tissues, that are thicker than the depth of field of a microscope objective, introduces a third dimension to a biopsy sample, in effect making the specimen, or portions thereof, three-dimensional in space. A consequence of the three-dimensional character of a specimen is that the cellular material is located at various focal planes, which thereby requires constant focusing and refocusing to observe cells at various contours of the sample.
Specifically, when obtaining a non-lacerational brush biopsy of a tissue, a brush is used which is sufficiently stiff so as to penetrate the various layers of epithelium In the process of obtaining a full thickness tissue specimen, tissue fragments in addition to single cells and cell clusters ate obtained and transferred onto a microscope slide. This occurs, when the brush biopsy instrument disclosed in U.S. Pat. No. 6,258,044, incorporated herein by reference is used to sample oral epithelial tissue. Similarly, when using the brush biopsy instruments for sampling tissue of the gastrointestinal tract disclosed in U.S. Pat. Nos. 6,494,845; 6,676,609 and 7,004,913, all of which are incorporated herein by reference, the resultant specimen contains single cells, cell clusters and thick tissue fragments. These specimens are markedly different from the cell monolayers prepared for the analysis of exfoliative cytological specimens, whereby only a superficial sweep of a tissue is conducted, and no tissue fragments are obtained.
This novel specimen, containing single cells, cell clusters and tissue fragments, is essentially a hybrid between a cytological smear, and histological sections. The ability to view tissue fragments, in addition to single cells, confers an enormous informational advantage to a pathologist in making a diagnosis. Intact tissue provides the pathologist with important information about a tissue's architecture, which is not available in cytological smears. This benefit is especially critical in the evaluation of gastrointestinal tissue, which is a complex tissue containing various cell types including glandular and columnar epithelium.
Furthermore, U.S. Pat. Nos. 6,297,044 and 6,284,482—both of which are incorporated herein by reference—disclose a computerized system for analyzing and classifying these novel specimens consisting of disaggregated cells and tissue fragments. In conducting its analysis of a slide, the computer scores and classifies cells that are most abnormal within a population of cells based on morphological criteria. As such, when analyzing a specimen that has dimensionality, often there will be areas of the specimen that are out of focus. As a result, the computer may classify cells found in these out of focus areas as abnormal because they may appear to the computer as exhibiting features that it is trained to classify as abnormal.
Composing a two-dimensional image out of the three-dimensional specimen would combine the respective advantages of each. A pathologist would be able to capture the information available from a three-dimensional sample without the drawbacks associated with the constant need to focus and refocus the microscope. This would additionally, make the computer analysis of such samples more sensitive, as normal cells that are out of focus and thusly appear to the computer as abnormal, would be eliminated or significantly reduced.
This can be achieved through extended depth of focus (EDF) processing techniques, which are well known in the prior art See, for example, U.S. Pat. No. 4,584,704. With EDF processing an automated microscope captures a set of images taken at different focal planes at the same location on a slide and then combines these images into a composite image. A single image appears sharp and well-focused only in those areas where the focal plane intersects the three-dimensional contours of the sample. A single high-resolution image cannot be in-focus everywhere. With EDF processing an automated microscope captures a set of images taken at regular z-intervals at the same location and then recovers from each slice those pixels that are in focus to build a single composite images from the in focus pixels. Once the in-focus pixels of each image are fused into one image, the resultant image is essentially in-focus everywhere.
However, the EDF algorithms of the prior art—which are ideal for creating a composite image of the top surface of a three-dimensional object—do not work well for biological specimens, which present an additional complexity of having multiple semi-transparent objects or cells stacked on each other. This is because in composing a composite image, standard EDF algorithms blindly extract the sharpest pixels from each focal plane, raising the possibility that a composite cell image contains pixels coming from multiple cells that happened to be situated on top of each other. In that instance a composite image may appear to represent a single cell, when in fact there were several cells stacked on top of each other, each of which could be observed by a microscope due to their semi-transparent character. For example, in U.S. Pat. No. 4,661,986 the best-focused pixels are selected to be incorporated into the composite image. In U.S. Pat. No. 7,058,233 a composite image is constructed by selecting well-focused edges or boundaries. These methods are of no utility when attempting to construct a composite image of a semi-transparent biological sample for diagnostic purposes, whereby it is critical to have all the pixels come from a single object or cell and to avoid pixel contamination. In order for an EDF algorithm to be properly applied to thick, semi-transparent biological specimens it must take into account which pixels belong to which object, and preserve most of the pixels of that cell or object even though they may not necessarily be sharp.
Additionally, for diagnostic purposes, a pathologist prefers to see a composite image that only includes cell nuclei This presents a further limitation of standard EDF algorithms; the nucleus' sharpness might be less than the cell boundary's sharpness, causing the composite image to only display the cell boundary and not the diagnostic important cell nucleus.
This problem is particularly pronounced in the case of columnar epithelial cells. Columnar cells, which are found in the lining of the gastro-intestinal tract, are characteristically tall or oblong with its nucleus usually situated in its middle to lower region. When viewing a sample of columnar cells that are vertically oriented on a slide, such that the microscope is looking down the axis of a cylindrical cell, the cell boundaries may be in best focus at the top of the sample while the nucleus is in-focus at the bottom. A pathologist needs to see the nucleus, but prior art EDF algorithms, which select in-focus images based solely on the sharpness of the image, cannot distinguish between different objects, both of which appear sharp and in-focus. As a result, both the cell boundary and the cell nucleus will be selected, or in some instances only the cell boundary will be selected. In the latter situation, the resultant composite image will feature an array of abutting cell boundaries, and will take on the appearance of fish scales. This image will be sharp and in-focus everywhere, but will not convey any diagnostically important information to an examining pathologist or to a specialized computer system that may be analyzing the slide.
In summary, the semi-transparent quality of biological specimens presents a problem with applying prior art EDF algorithms to thick cellular specimens. The prior algorithms were designed to create a composite image of the outside surface of a three dimensional object. With biological specimens, on the other hand, the area of interest is not limited to the outside surface, but rather there are areas of interest below the outside surface of a specimen. Due to light absorption within a tissue, structures on the surface have a tendency to have a higher contrast or sharpness than structures inside a transparent tissue. Because tissue is comprised of a mass of cells, a pathologist needs to see the cells and nuclei that are situated below the outside surface, which comprise the thickness of the tissue. Additionally due to the disaggregated nature of the cellular samples obtained by a brush biopsy, a slide prepared in connection with a brush biopsy will inevitably contain cells that overlap and overlay each other. Consequently, an examining pathologist or specialized computer would want to build a constraint into the EDF algorithm such that pixels of objects can only come from one object or cell only and not from neighboring objects that are situated either on top or beneath the cell being imaged. For the purpose of this document this requirement is referred to as “PCO” or Pixel Constrained to an Object”. Furthermore, there are objects of no diagnostic significance, such as cell walls or artifacts that should be deemphasized or eliminated from a composite image.